10/522590 

DT05 Rec'd PCT/PTO 2 6 JAN 2005 

wo 2004/017252 PCT/DE2003/002467 



Specification 

Method for Evaluating the Signals of an Electronic Image 
Sensor During Pattern Recognition of Image Contents of a Test 
Piece 

The invention relates to methods for signal evaluation 
of an electronic image sensor in the course of pattern 
recognition of the image contents of a test body in 
accordance with the preambles of claims 1, 2 or 19. . 

Known methods for analyzing the image contents of a 
test body are mainly based on metrics for determining 
similarities, such as distance measurements of segmented 
objects, or the calculation of global threshold 
distributions. These methods are based on translatorily 
invariable initial spectra. Situations often occur in 
reality, such as object displacements underneath the 
recording system, or different backgrounds during recording, 
or aliasing effects, so that in many cases a direct 
comparison of these initial spectra cannot be performed. 

It is known from the reference book of Thomas TILLI, 
"Mustererkennung mit Fuzzy-Logik: Analysieren, 
klassif izieren, erkennen und diagnostizieren" [Pattern 
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Recognition by Means of Fuzzy Logic: Analyzing, Classifying, 
Determining and Diagnosing] , Franzis-Verlag GmbH, Miinchen, 
publishers, 1993, pp. 183/184, 208 to 210, 235 to 257, to use 
fuzzy logic for image processing, wherein a spectral 
transformation can be one type of signal preparation. 

In the technical article "Mustererkennung mit Fuzzy- 
Logik" [Pattern Recognition by Means of Fuzzy Logic] by Peter 
ARNEMANN, Elektronik 22/1992, pages 88 to 92, it is described 
how to perform pattern recognition by means of fuzzy logic. 

The article by D. Charalampidis, T. Kasparis, M. 
Georgiopoulos, J. Rolland "A Fuzzy ARTMAP-Based 
Classification Technique of Natural Textures", Fuzzy 
Information Processing Society, 1999, NAFIPS, 18th 
International Conference of the North American, June 10 to 12 
1999, pp. 507 to 511, describes the performance of pattern 
recognition with a training phase and the use of a window of 
16 X 16 pixels for image recognition. 

The object of the invention is based on providing 
methods for signal evaluation of an electronic image sensor 
in the course of pattern recognition of the image contents of 
a test body. 
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In accordance with the invention, this object is 
attained by the characteristics of claims 1, 2 or 19. 

An advantage of the invention lies in particular in 
that a sensor signal is analyzed in an image window of the 
size of n X n pixels. As a result of this it is possible to 
consider the sensor signal of this image window to be local. 
The image analysis method in accordance with the invention 
can be divided into the substantial steps: characteristics 
formation, fuzzyfying, interference, defuzzyfying and 
decision regarding the class affiliation. 

In the course of characteristics formation, the sensor 
signal is converted by means of at least one calculation 
specification into an invariant, in particular translation- 
invariant, signal in the characteristic space. It is the aim 
of the characteristics formation to define those values by 
means of which typical signal properties of the image content 
are characterized. The typical signal properties of the 
image content are represented by so-called characteristics. 
In this case the characteristics can be represented by values 
in the characteristic space, or by linguistic variables. A 
signal is created by transferring the sensor signal into the 
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characteristic space, which consists of one characteristic 
value or several characteristic values. 

The affiliation of a characteristic value with a 
characteristic is described by at least one indistinct 
affiliation function. This is a soft or indistinct 
affiliation, wherein the affiliation of the characteristic 
value with the characteristic exists as a function of the 
characteristic value in a standardized interval between 0 and 
1. The concept of the affiliation function leads to a 
characteristic value no longer totally, or not at all, being 
capable of being affiliated with a characteristic, but 
instead can take on a fuzzy affiliation, which is located 
between the Boolean logical functions 1 and 0. The just 
described step is called f uzzyf ication . Thus, in the course 
of fuzzyf ication a conversion of a definite characteristic 
value into one or several indistinct affiliations takes 
substantially place. 

In connection with the interference, a higher order 
affiliation function is generated by means of a calculation 
specification consisting of at least one rule, wherein all 
affiliation functions are linked to each other. As a result. 
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a higher order affiliation function is therefore obtained for 
each window. 

In connection with def uzzyf ication, a number value, 
also called a sympathetic value, is determined from the 
higher order affiliation function formed during interference. 
In the course of the decision regarding class affiliation, a 
comparison of the sympathetic value with a previously fixed 
threshold value takes place, by means of which the 
affiliation of the window with a defined class is decided. 

What type the characteristic values in the 
characteristic space are is of lesser importance for the 
principle of the invention. Thus, for example, in connection 
with time signals there is the possibility to set the mean 
value or the variance as characteristic values. If it is 
required of the evaluation process that it can process the 
image contents free of errors regardless of the respectively 
prevailing signal intensity, and if furthermore small, but 
permissible fluctuations in the image signal do not lead to 
interference, it is useful if the conversion of the sensor 
signal from the two-dimensional local space is performed by 
means of a two-dimensional spectral transformation, such as. 
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for example, a two-dimensional Fourier, or a two-dimensional 
Walsh, or a two-dimensional Hadamard, or a two-dimensional 
circular transformation. Invariant characteristic values are 
obtained by means of the two-dimensional spectral 
transformation. A further preferred embodiment consists in 
using the amount of the spectral coefficient obtained by the 
spectral transformation as the characteristic value. 

In a preferred exemplary embodiment, the affiliation 
functions are unimodal potential functions, and the higher 
order affiliation function is a multimodal potential 
function. 

In accordance with a further preferred exemplary 
embodiment, at least one affiliation function is 
parametrized. If the affiliation function has positive and 
negative slopes, it is advantageous if it is possible to 
determine the positive and negative slopes separately. An 
improved matching of the parameters to the data sets to be 
examined is assured by this. 

In accordance with a particularly preferred exemplary 
embodiment, the method can be divided into a learning phase 
and a working phase. If the affiliation functions are 
parametrized, it is possible in the learning phase to 
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determine the parameters of the affiliation functions from 
measured data sets. In the learning phase, the parameters of 
the affiliation functions are adapted to so-called reference 
images, i.e. during the learning phase an affiliation of the 
characteristic values resulting from the reference images 
with the respective characteristics is derived by means of 
the affiliation functions and their parameters. In the 
subsequent work phase the characteristic values resulting 
from the now measured data sets are weighted with the 
affiliation functions whose parameters had been determined in 
the learning phase, from which an affiliation of the 
characteristic values of the now measured data sets with the 
corresponding characteristics is produced. By dividing the 
method into a learning and a work phase the parameters of the 
affiliation functions are determined by means of measured 
reference data sets, and in the subsequent work phase the 
measured data sets, which are to be tested, are weighted with 
the affiliation functions fixed during the learning phase and 
are evaluated. 

In accordance with a further preferred exemplary 
embodiment at least one rule by means of which the 
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affiliation functions are linked with each other, is a 
conjunctive rule within the meaning of an IF ...THEN linkage. 

A further preferred exemplary embodiment subdivides the 
generation of the higher order indistinct affiliation 
functions into the processing of the partial steps: premise 
evaluation, activation and aggregation. In this case, in the 
premise evaluation an affiliation value is determined for 
each IF portion of a rule, and during the activation an 
affiliation function is fixed for each IF . . . THEN rule. 
Thereafter, during the aggregation, the higher order 
affiliation function is generated by superimposing all 
affiliation functions created during the activation. 

In accordance with a further preferred exemplary 
embodiment, the sympathetic value determination is performed 
in particular in accordance with a main emphasis and/or 
maximum method. 

Exemplary embodiments of the invention are represented 
in the drawings and will be described in greater detail in 
what follows. 

Shown are in: 

Fig. 1, a flow diagram of the signal evaluation method. 
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Fig. 2, a sympathetic curve. 

Fig. 3a, a difference function of the power of D = 8 
Fig. 3b, a difference function of the power of D = 4 
Fig. 3c, a difference function of the power of D = 2 
A flow diagram of the signal evaluation method to be 
described in what follows is shown in Fig. 1. With the 
method for signal evaluation of image contents of a test 
body, a grid of N x N windows 01 is placed over the entire 
image to be analyzed. Each window 01 here consists of n x n 
pixels 02. In the course of the image analysis, the signal 
from each window 01 is analyzed separately. As a result, the 
image content 03 of each window 01 can be considered to be 
local . 

The two-dimensional image of the local space is 
transformed into a two-dimensional image in the frequency 
space by one or several spectral transformations. The 
spectrum obtained is called a frequency spectrum. Since this 
is a discrete spectrum in the present exemplary embodiment, 
the frequency spectrum is also discrete. The frequency 
spectrum is constituted by the spectral coefficients 06 - 
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also called spectral values 06 

In the next method step the amount formation 07 of the 
spectral values 06 takes place. The amount of the spectral 
values 06 is called spectral amplitude value 08. In the 
present exemplary embodiment, the spectral amplitude values 
08 constitute the characteristic values, i.e. they are 
identical to the characteristic values. 

A circular transformation is preferably used for the 
transformation. With the circular transformation the 
invariance properties can be adjusted via the transformation 



CORRECTED PAGE (RULE 91) 



wo 2004/017252 



PCT/DE2003/002467 



coefficients. It is possible to set a translation 
invariance, as well as a reflection invariance, or an 
invariance in respect to different other permutation groups. 
In this way it is possible to utilize the above mentioned 
transformation for example in the reflection-variant 
variation for inspecting characters 
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(consider the differentiation between the numbers "9" and 
"6"). In the same way the reflection-invariant variation can 
be used for inspecting workpieces, since here it is not 
necessary in particular to make a differentiation between a 
reflected part and the original. It should be mentioned that 
the amount spectrum of the Fourier transformation is 
reflection-invariant . 

These transformations work with real coefficient 
values. It is therefore not necessary to utilize a complex 
calculation as with the Fourier transformation. 

The circular transformation is extremely tolerant even 
in the sub-pixel range in connection with any arbitrary 
displacements. Comparisons have shown that this circular 
transformation is superior to other known transformations in 
regard to displacements. 

The number of work coefficients (characteristics, 
fea.tures) is small, because the spectral coefficients are 
again combined in groups . 

The tolerance to displacements is created by the 
combination. Even if a signal runs partially out of a 
measurement field, the characteristics remain relatively 
stable. Tests have shown that stability is maintained, even 



wo 2004/017252 



PCT/DE2003/002467 



if the image contents lie outside of the measurement field by 
up to approximately 3 0%. 

The characteristic selection 09 follows as a further 
method step; the aim of the characteristic selection 09 is to 
select the characteristics 11, which are characteristic for 
the image content 03 of the image to be analyzed. 
Characteristic spectral amplitude values 08, which define the 
characteristic 11 by their position in the frequency space 
and by their amplitude, are possible as characteristics 11, 
but also linguistic variables such as "gray", "black" or 
"white" . 

In the next following method step, the f uzzyf ication 
12, the affiliation of each spectral amplitude value 08 with 
a characteristics 11 is fixed by means of a soft or 
indistinct affiliation function 13, i. e. weighting is 
performed. 

If it is intended during a learning phase to match the 
affiliation functions 13 to so-called reference data sets, it 
is useful if the affiliation functions 13 are parametrized 
monomodal, i.e. one -dimensional potential functions, wherein 
the parameters of the positive and negative slopes can be 
matched separately to the data sets to be examined. In the 
work phase which follows the learning phase, the data sets of 
the image content from which the characteristic values 08 of 
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the test images result are weighted with the respective 
affiliation functions 13 whose parameters had been determined 
in the previous learning phase. This means that for each 
characteristic 11 a sort of TARGET-ACTUAL comparison between 
the reference data set, expressed in the parameters of the 
affiliation function 13, and the data set of the test image 
takes place. A soft or indistinct affiliation between the 
respective characteristic value 08 and the characteristic 11 
is made by means of the affiliation functions 13. 

In the next method step, the interference 14, a 
conjunctive linkage 15 - also called aggregation 15 - of all 
affiliation functions 13 of the characteristics 11 takes 
place, so that a higher order affiliation function 16 is 
created. 

The next method step, the def uzzyf ication 17, 
determines a concrete affiliation or sympathetic value 18 
from the higher order affiliation function 16. During the 
classification 19, this sympathetic value 18 is compared with 
a previously set threshold value 21, so that a classification 
statement can be made. The threshold value 21 is set either 
manually or automatically. Setting of the threshold value 21 
takes also place during the learning phase. 

During the classification, a numerical value is not 
assigned directly to a defined class by means of a true or 
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false statement, a unimodal function is set instead, which 
describes an affiliation with a true or false statement. 

In the course of this the class affiliation is trained, 
i.e. the decision curves are taught by means of measured 
values determined during the process. The functions by means 
of which a degree of affiliation is determined, are called 
affiliation functions ZGF = /x (m^) . The calculated value of 
the affiliation function ZGF is called sympathetic value /x. 
Several affiliation functions ZGF. are often used, which are 
further combined in the subsequent steps in order to achieve 
an unequivocal statement . 

However, this is specifically not a neuronal network 
being used. It is known that neuronal networks can be 
trained. 

The fuzzy plate classification is based on a concept 
which simultaneously provides a distance measurement and a 
characteristic linkage. The "fuzzy" fact here is that the 
characteristics are "rounded off", not logically, but 
indistinctly. For one, this leads to all characteristics 
being summarily considered. This means that small deviations 
from a characteristic are still tolerated. If secondly the 
deviation from a characteristic becomes too large, this has 
immediately a large effect on the distance measurement. 
Accordingly, the output of the classif icator does not provide 
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a "good/bad" decision, but a continuous output value between 
[0 ... 1] . Thereafter a threshold value is used, which makes 
a "good/bad" decision possible. 

The output value for the distance measurement 
(sympathetic value) is fx = 2"^, wherein 




Here, the coefficients have the following meanings: x = 
counting index, z = averaged distance measurement, M = number 
of characteristics, Xq = mean value of C^iff , Cx = expansion 
value, D = power, ijl = sympathetic value,. C^iff = difference 
measurement of the expansion value. 

The expansion value C is taught with the aid of 
measured values which had been generated by means of the 
circular transformation. 

The /i -value describes how close the similarity of a 
pattern is in relation to a reference pattern described by 
the characteristics. This means that the z-value takes over 
the actual control of the /z-value. If the z-value is very 
small, the /i-value is close to 1. The patterns are very 
similar (sympathetic) . However, if the z-value is large, the 
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/x-value will become small, the patterns are not similar. The 
course of the curve - as implemented - is represented in Fig. 
2. 

Initially, in the learning phase the values Cdiff^ are 
determined, namely for each characteristic m^^ one value 

Cdiffx = max(mx) - min(mx) 

wherein C^iff is the difference measurement of the expansion 
value, and m the characteristics. 

During the inspection the learned C^iff values are 
used. The values can still be assigned an additional 
tolerance a. Settlement takes place during the running time: 
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max(mx) - min(mx) 
Cx = (1 + 2 Pee) , a = (1 + 2 p^e) 



wherein C is the expansion value and P^e is the percental 
tolerance of C^iff. 

The value range of a lies between [1 ... 3] . The value 
Pqq indicates the percental tolerance with which C^iff is 
respectively charged. A 50% expansion of the range of C^iff 
is intended to be achieved; in that case a = 1+2* 0 . 5 = 2. 

The Xq value indicates the mean value of C^^ff ; it is 
calculated for each characteristic during the running time. 

The difference between the characteristic value and the 
mean characteristic value, which is determined from the value 

is calculated. This difference is standardized with the 
width of the expansion value Cx- The result is that with a 
slight deviation the corresponding characteristic contributes 
little to the z -value; however, with a large deviation a 
large deviation value will result as a function of the 
difference measure of the expansion value C^jiff . The 
standardized difference is called dx- 

The power D (2, 4, 8) sets the sensitivity at the 
flanks of the standardized difference function dx- If the 
value D is set to "infinity" - which is not technically 
possible - an infinite flank steepness is also obtained, and 
therefore a hard good/bad decision. Therefore the values are 
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customarily set to between 2 ... 20. The curves for the 
values 2, 4 and 8 are represented in Figs. 3c, 3b and 3a. 

The exponentiated functions d^ are added up, however, 
only the number M of the characteristics m which have been 
switched on is used. Following the adding-up, the calculated 
value is divided by the number M. The mean value of all 
exponentiated differences d^ is determined. 
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The effect is the following: because of the 
exponentiation, small deviations will not be important, but 
large ones will be increased. A deviation of all 
characteristic differences is calculated by averaging. This 
has the result that even with the deviation of several 
characteristics the /x-value is not drastically lowered. This 
value will become very small only with larger deviations. 

A threshold value evaluation follows thereafter. 

Good, if /X (z) s/ig 

Mklass = 

Error, if /x (z) </Xs 

This process is performed for all windows. 

An evaluation of dynamic processes - such a printing 
processes - requires non-linear distance measurements 
(sympathetic values) . 
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List of Reference Symbols 



01 Window, N x N windows 

02 n X n pixels 

03 Image contents 

04 2 -dimensional spectral transformation, 
calculation specification 

05 

06 Spectral coefficient, spectral value 

07 2 -dimensional amount formation, calculation 
specification 

08 Spectral amplitude value=characteristic value 

09 Characteristics selection 
10 

11 Characteristic 

12 Fuzzyf ication 

13 Affiliation function 

14 Interference, calculation specification 
15 

16 Higher order affiliation function, linkage, 
aggregation 

17 Defuzzyf ication 

18 Affiliation value, sympathetic value 

19 Classification, class affiliation 
20 

21 Threshold value 

C Expansion value 

^diff Difference measure of the expansion value 

D Power 

M Number of characteristics 
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ZGF Affiliation function 

a Tolerance 

dx Nominated difference 

m Characteristic 

Pee Percental tolerance of C^iff 

X Counting index 

z Averaged distance measurement 

II Sympathetic value, distance measurement 



